Description

Fixes #31227 - Resolves the issue where OpenAIEmbeddings exceeds OpenAI's 300,000 token per request limit, causing 400 BadRequest errors.

Problem

When embedding large document sets, LangChain would send batches containing more than 300,000 tokens in a single API request, causing this error:

openai.BadRequestError: Error code: 400 - {'error': {'message': 'Requested 673477 tokens, max 300000 tokens per request'}}

The issue occurred because:

• The code chunks texts by embedding_ctx_length (8191 tokens per chunk)
• Then batches chunks by chunk_size (default 1000 chunks per request)
• But didn't check: Total tokens per batch against OpenAI's 300k limit
• Result: 1000 chunks × 8191 tokens = 8,191,000 tokens → Exceeds limit!

Solution

This PR implements dynamic batching that respects the 300k token limit:

1. Added constant: MAX_TOKENS_PER_REQUEST = 300000
2. Track token counts: Calculate actual tokens for each chunk
3. Dynamic batching: Instead of fixed chunk_size batches, accumulate chunks until approaching the 300k limit
4. Applied to both sync and async: Fixed both _get_len_safe_embeddings and _aget_len_safe_embeddings

Changes

• Modified langchain_openai/embeddings/base.py:
  • Added MAX_TOKENS_PER_REQUEST constant
  • Replaced fixed-size batching with token-aware dynamic batching
  • Applied to both sync (line ~478) and async (line ~527) methods
• Added test in tests/unit_tests/embeddings/test_base.py:
  • test_embeddings_respects_token_limit() - Verifies large document sets are properly batched

Testing

All existing tests pass (280 passed, 4 xfailed, 1 xpassed).

New test verifies:

• Large document sets (500 texts × 1000 tokens = 500k tokens) are split into multiple API calls
• Each API call respects the 300k token limit

Usage

After this fix, users can embed large document sets without errors:

from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma
from langchain_text_splitters import CharacterTextSplitter

# This will now work without exceeding token limits
embeddings = OpenAIEmbeddings()
documents = CharacterTextSplitter().split_documents(large_documents)
Chroma.from_documents(documents, embeddings)

Resolves #31227